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(S) Apparatus and method for defocusing images. 



(S?) The present invention discloses an image defocusing apparatus for defocusing an image formed on a 
screen by modeling an object in a boundary box by updating pixel values on the image to approximate a 
lens's focusing effects and a method thereof. The apparatus comprises a unit for computing data 
related to a circle of confusion for each point in the boundary box, the circle of confusion being a 
measure how defocused an out-of-focus point is on the screen, a unit for checking a reference pixel 
whose corresponding point's circle of confusion overlaps a pixel selected as a target pixel, and a unit 
for updating an original value of the target pixel by taking into account a value of the reference pixel and 
the data of the target pixel's circle of confusion. 
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BACKGROUND OF THE INVENTION 

(1) Field of the Invention 

5 The present invention relates to an image defocusing apparatus capable of applying a real lens's focusing 

effects on an image formed by computer graphics, and to a method thereof. 

(2) Description of the Related Art 

10 The rapid progress in the field of computer graphics makes it possible to synthesize an image formed by 

the computer graphics into a video taken by a video camera. The synthetic image, however, will be unrealistic 
unless a depth distance is taken into account, for example, by a process called "defocusing". 

With the "defocusing", an object in focus appears clear and sharp while the front and rear of the object, 
or out-of-focus points, fuzzy and blurred in a resulting image in an approximate way when seen through a carrv 

15 era. (Rendering an image by modeling an object with a boundary box is a known art and the explanation thereof 
is omitted herein.) 

Typically, a method called "ray-tracing" is used for the "defocusing". However, the "ray-tracing" or other 
applicable methods involve a considerable amount of computation, and thus the entire process takes quite a 
long time. To solve this problem, a simpler, faster method has been disclosed in Japanese Laid-Open Patent 
20 Application No. 63-259778. In this method, a region of defocus for each sample point is determined with their 
respective Z values that represent a distance from a view point to each sample point, and a focal point of a 
lens being used; the intensities within the region of defocus are averaged to update the original intensities with 
the mean intensity. 

The more detailed explanation will be given by referring to FIG. 1 , which shows the relation between the 
25 Z values and regions of defocus relative to the focal point Here, a view point is fixed at z = 0 coordinate, and 
several points, Za, Zb, Zc(focal point), Zd, and Ze, are fixed along the z coordinate, the direction in which an 
eye ray is aimed. Assume that Z is the z coordinate of the sample point, i.e. the Z value, then the region of 
defocus of the sample point is determined as follows: 



30 


Conditions 


Region of Defocus 




1)Zd^Z>Zb 


none 




2) Ze^Z>Zd orZbi=Z>Za 


3x3 


35 


3) Z>Ze or Za^Z 


5x5 



In the drawing, a square 501 represents a pixel corresponding to the sample point, while squares 502, 
503 the regions of defocus under the conditions 1), 2), respectively. 

Given the region of defocus, either an arithmetic mean or a weighted mean of the intensities is computed, 
40 and the original intensities of all the pixels within the region of defocus are updated by the mean intensity. Since 
the area of the region of defocus is proportional to a distance from the focal point, a viewer can see the depth 
distance in the resulting defocused image. 

By exploiting the mean intensity as has been described, the amount of the computation is reduced, but 
only at the expense of degradation in image quality. 
45 For example, when a model shown in FIG. 2A- a red object with a white background placed in the boundary 

box - is rendered to an image shown in FIG. 2B and thence to a defocused image, the result will be the one 
shown in FIG. 2C. 

This is because the mean intensity of the region of defocus a2 for the sample point al naturally includes 
the red intensity since a2 overlaps the pixels on the object's image. This causes the edge of the object to appear 
50 fuzzy and blurred, or the object to appear larger than it should in the image: a phenomenon that never occurs 
when seen through a camera. A similar phenomenon does occur even when the object is out of focus as long 
as there is the depth distance in the model. 

SUMMARY OF THE INVENTION 

55 

Accordingly, the present invention has a first object to provide an image defocusing apparatus capable 
of distributing no intensity(pixel value) on an object to its background in a defocused image whether the object 
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is in focus or out of focus, and capable of computing an intensity distribution without decreasing the speed 
realized to date. 

The present invention has a second object to provide an image defocusing apparatus capable of forming 
the object's image of a size it should be in the defocused image. 
5 The present invention has a third object to provide an image defocusing apparatus capable of approxi- 

mating a real lens's focusing effects on the defocused image. 

The present invention has a fourth object to provide an image defocusing apparatus capable of realizing 
a depth distance corresponding to the Z value given to each pixel in the defocused image. 

The present invention has a fifth object to provide a rational method of defocusing an image applied to 
10 the above image defocusing apparatuses. 

The first through fourth objects are fulfilled by an image defocusing apparatus for defocusing an image 
formed on a screen by modeling an object in a boundary box by updating pixel values on the image to approx- 
imate a lens's focusing effects. The apparatus comprises: a unit for computing data related to a circle of con- 
fusion for each point in the boundary box, the circle of confusion being a measure how defocused an out-of- 
15 focus point is on the screen; a unit for checking a reference pixel whose corresponding point's circle of confu- 
sion overlaps a target pixel; and a unit for updating an original value of the target pixel by taking into account 
a value of the reference pixel and the data related to the target pixel's circle of confusion. 

The data related to the circle of confusion may comprise a radius and an area of each circle of confusion. 
The circle-of-confusion computing unit may include a radius calculating unit for computing a radius C of each 
20 circle of confusion using an expression: 

C = |1 -p(d-f)/d(p-f)l * f/F 
where p is a distance from a view point to a focal point, d is a Z value that represents a distance from 
the view point to each point, f is a focal length, and F is an iris value of a lens being used. The updating unit 
may include an area calculating unit for computing an area of each circle of confusion using their respective 
25 radiuses C. 

The circle-of-confusion computing unit may further include a storage unit for storing the radiuses C of the 
circles of confusion in relation with their respective pixels' x, y coordinates. 

The checking unit may include: a distance computing unit for computing a linear distance from the target 
pixel to each neighboring pixel; a comparing unit for comparing the linear distance with the radius C relative 
30 to each neighboring pixel by referring to their x, y coordinates; and a comparison-result outputting unit for se- 
lecting a neighboring pixel whose radius C is longer than the linear distance, and subsequently outputting the 
selected neighboring pixel as the reference pixel. 

The updating unit may include a distributing unit for weighting a value of each reference pixel to determine 
a distribution within their own circles of confusion, the distribution being distributed to the target pixel; a target- 
35 pixel weighting unit for weighting an original value of the target pixel with the area of its circle of confusion; 
and an updating device for accumulating the distributions and the weighted value of the target pixel to yield a 
mean value, the original value of the target pixel being updated by the mean value. 

The distributing unit may include a reference- pixel weighting unit for multiplying a value of each reference 
pixel by a reciprocal of an area of their respective circles of confusion. 
40 The checking unit may further include: a spatial-location comparing unit for comparing Z values of the tar- 

get pixel and the reference pixels, the Z value representing a distance from the view point to their corresponding 
points; and the image defocusing apparatus may further comprise: a unit for disallowing the updating unit to 
update the original value of the target pixel when the reference pixel has a larger Z value than the target pixel. 

The checking unit may further include an output unit for comparing the Z values of the target pixel and the 
45 neighboring pixels to select a neighboring pixel whose Z value is smaller than the target pixel, and for subse- 
quently outputting the selected neighboring pixel to the distance computing unit, the Z value representing a 
distance from the view point to their corresponding points. 

The first through fourth objects are also fulfilled by an image defocusing apparatus for defocusing an im- 
age formed on a screen by modeling an object in a boundary box by updating pixel values on the image to 
50 approximate a lens's focusing effects. The apparatus comprises: a unit for computing data related to a circle 
of confusion for each point in the boundary box, the circle of confusion being a measure how defocused an 
out-of-focus point is on the screen; a first checking unit for checking a first reference pixel whose corresponding 
point's circle of confusion overlaps a target pixel in a first direction; a first updating unit for updating an original 
value of the target pixel by taking into account a value of the first reference pixel and the data related to the 
55 target pixel's circle of confusion; a second checking unit for checking a second reference pixel whose corre- 
sponding point's circle of confusion overlaps the target pixel in a second direction, the second direction being 
orthogonal with respect to the first direction; and a second updating unit for updating the values updated by 
the first updating unit by taking into account a value of the second reference pixel and the data related to the 
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target pixel's circle of confusion. 

The data related to the circle of confusion may comprise a radius and an area of each circle of confusion. 
The circle-of-conf usion computing unit may include a radius calculating unit for computing a radius C of each 
circle of confusion using an expression: 
5 C = 1 1 -p(d-f)/d(P-f)l * f/F 

where p is a distance from a view point to a focal point, d is a 2 value that represents a distance from 
the view point to each point, f is a focal length, and F is an iris value of a lens being used. The updating unit 
may include an area calculating unit for computing an area of each circle of confusion using their respective 
radiuses C. 

10 The circle-of-conf usion computing unit may further include a storage unit for storing the radiuses C of the 

circles of confusion in relation with their respective pixels' x, y coordinates. 

The first checking unit may include: a first distance computing unit for counting the number of pixels be- 
tween the target pixel and each pixel aligned in an array that contains the target pixel in the first direction, the 
number thus counted being a distance; a first comparing unit for comparing the distance and the radius C rel- 

15 ative to each pixel aligned in the target pixel's array in the first direction by referring to their x, y coordinates; 
a first comparison-result outputting unit for outputting a pixel aligned in the first direction whose radius C is 
longer than the distance to the first updating unit as the first reference pixel. The second checking unit may 
include: a second distance computing unit for counting the number of pixels between the target pixel and each 
pixel aligned in an array that contains the target pixel in the second direction, the number thus counted being 

20 a distance; a second comparing unit for comparing the distance and the radius C relative to each pixel aligned 
in the target pixel's array in the second direction by referring to their x, y coordinates; and a second comparison- 
result outputting unit for outputting a pixel aligned in the second direction whose radius C is longer than the 
distance to the second updating unit as the second reference pixel. 

The first updating unit may include a first distributing unit for weighting a value of the first reference pixel 

25 to determine a distribution within their own circles of confusion, the distribution being distributed to the target 
pixel. The second updating unit may include a second distributing unit for weighting a value of the second ref- 
erence pixel to determine a distribution within their own circles of confusion, the distribution being distributed 
to the target pixel. 

The first distributing unit may include a first reference-pixel weighting unit for multiplying an original value 
30 of each reference pixel by a reciprocal of an area of their respective circles of confusion, and the second dis- 
tributing unit may include a second reference- pixel weighting unit for multiplying an original value of each ref- 
erence pixel by a reciprocal of an area of their respective circles of confusion. 

The image defocusing apparatus may further comprise a unit for withholding the value of the target pixel 
updated by the first updating unit to transfer them to the second updating unit 
35 The first checking unit may include a first spatial-location comparing unit for comparing the Z values of 

the target pixel and the first reference pixel, the Z value representing a distance from the view point to their 
corresponding points. The second checking unit may include a second spatial-location comparing unit for com- 
paring the Z values of the target pixel and the second reference pixel. The first updating unit may include a 
first disallowing unit for disallowing the first updating unit to update the original value of the target pixel when 
40 the target pixel has a smaller Z value. The second updating unit may include a second disallowing unit for dis- 
allowing the second updating unit to update the original value of the target pixel when the target pixel has a 
smaller Z value. 

The first checking unit may further include a first output unit for comparing the Z values of the target pixel 
and the neighboring pixels to select a neighboring pixel whose Z value is smaller than the target pixel, and for 

45 subsequently outputting the selected neighboring pixel to the first distance computing unit, the Z value rep- 
resenting a distance from the view point to their corresponding points. The second checking unit may further 
include a second output unit for comparing Z values of the target pixel and the neighboring pixels to select a 
neighboring pixel whose Z value is smaller than the target pixel, and for subsequently outputting the selected 
neighboring pixel to the second distance computing unit. 

50 The fifth object is fulfilled by a method of defocusing an image by updating pixel values on the image 

formed on a screen by modeling an object in a boundary box. The method comprises the steps of: 

(1) computing data related to a circle of confusion for each point in the boundary box, the circle of confusion 
being a measure how defocused an out-of- focus point is on the screen; 

(2) checking a reference pixel whose corresponding point's circle of confusion overlaps a target pixel; and 
55 (3) updating an original value of the target pixel by taking into account a value of the reference pixel and 

the data related to the target pixel's circle of confusion. 
The first step may comprise the substep of: 

(a) computing a radius C of each circle of confusion using an expression: 
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C = 1 1 - p(d - f)/d(p - f) I * f/F 
where p is a distance from a view point to a focal point, d is a Z value, f is a focal length, and F is an iris 
value of a lens being used. 

The second step may include the sub-steps of: 
5 (a) computing a distance from the target pixel to each neighboring pixel; 

(b) comparing the distance with a radius C of the overlapping circle of confusion relative to each neigh- 
boring pixel; 

(c) selecting a neighboring pixel whose radius C is longer than the distance as the reference pixel; 

(d) selecting another target pixel; and 

10 (e) proceeding the third step when all the pixels on the image have been selected as the target pixel and 

been through the substeps (a) through (c) of the second step. 
The third step may comprise the substeps of. 

(a) weighting a value of each reference pixel to determine a distribution to the target pixel; 

(b) computing an area of each overlapping circle of confusion and the target pixel's circle of confusion to 
15 weight the original value of the target pixel with the area of its circle of confusion; 

(c) accumulating the distributions and the weighted value of the target pixel to yield a mean value, and 
updating the original value of the target pixel with the mean value; 

(d) selecting another target pixel; and 

(e) terminating to update the target pixel's value when all the pixels have been selected as the target pixel 
20 and been through the substeps (a) through (c) of the third step. 

The image defocusing method may further comprise the steps of: 

(4) comparing Z values of the target pixels and neighboring pixels, the Z value representing a distance 
from a view point to their corresponding points, and 

(5) disallowing to update the original value of the target pixel when the neighboring pixel has a larger Z 
25 value than the target pixel. 

The second step may further comprise the substep of: 

(f) comparing Z values of the target pixels and neighboring pixels to select a neighboring pixel whose Z 
value is smaller than the target pixel prior to the substep (a) of the second step. 

The fifth object is also fulfilled by a method of defocusing an image by updating pixel values on the image 
30 formed on a screen by modeling an object in a boundary box. The method comprises the steps of: 

(1) computing data related to a circle of confusion for each point in the boundary box, the circle of confusion 
being a measure how defocused an out-of-focus point is on the screen; 

(2) checking a first reference pixel whose corresponding point's circle of confusion overlaps a target pixel 
along a first direction; 

35 (3) updating an original value of the target pixel by taking into account a value of the first reference pixel 

and the data related to the target pixel's circle of confusion; 

(4) checking a second reference pixel whose corresponding point's circle of confusion overlaps the target 
pixel along a second direction, the second direction being orthogonal with respect to the first direction; 
and 

40 (5) updating the value updated at the third step by taking into account a value of the second reference 

pixel and the data related to the target pixel's circle of confusion. 
The second and fourth steps may respectively comprise the substep of: 

(a) comparing Z values of the target pixels and neighboring pixels, the Z value representing a distance 
from a view point to their corresponding points. The third and fifth steps may respectively comprise the substep 
45 of: 

(a) disallowing to update the original value of the target pixel when the neighboring pixel has a larger Z 
value than the target pixel. 

The second and fourth steps may further and respectively comprise the substep of: 

(b) comparing the Z values of the target pixels and neighboring pixels to select a neighboring pixel whose 
50 Z value is smaller than the target pixel prior to the substep (a) of the second step and the substep (a) of the 

fourth step, respectively . 

According to the present invention, data related to the circle of confusion of each pixel is computed by tak- 
ing into account their respective Z values and the focal length and focal point of the lens being used. As a result, 
the color intensity(pixel value) on the object in focus is not distributed to the background in the defocused im- 
55 age, disallowing the object to appear larger than it should. Also, the color intensity on the object out of focus 
is distributed according to the Z values and lens's focusing effects. Thus, the defocused image approximates 
the way they appear when seen through a camera. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

These and other objects, advantages and features of the invention will become apparent from the following 
description thereof taken in conjunction with the accompanying drawings which illustrate specific embodi- 
5 ments of the invention. In the drawings: 

FIG. 1 is a view explaining a conventional "defocusing" process; 

FIG. 2 is an example of a defocused image formed by the conventional "defocusing" process; 
FIG. 3 is a view showing a structure of an image defocusing apparatus of the present invention; 
FIG. 4 is a perspective view showing the relation between a boundary box and a screen in location; 
10 FIG. 5 is a block diagram depicting the structure of a defocus processing unit in accordance with the first 

embodiment; 

FIG. 6 is an illustration of examples of the data in an image forming buffer, an original-image saving buffer, 
a Z buffer, and a COC buffer; 

FIG. 7 is an illustration explaining the selection of reference pixels by a RP address calculator, 
15 FIG. 8 is a flowchart detailing the operation of the defocus processing unit in FIG. 5; 

FIG. 9 is sub-routine of the flowchart in FIG. 8; 

FIG. 10 is a block diagram depicting the structure of a defocus processing unit in accordance with the sec- 
ond embodiment; 

FIG. 11 is an illustration explaining the selection of reference pixels by a first RP address calculator; 
20 FIG. 12 is an illustration explaining the selection of reference pixels by a second RP address calculator; 

and 

FIG. 13 is a flowchart detailing the operation of the defocus processing unit in FIG. 10. 
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DESCRIPTION OF THE PREFERRED EMBODIMENTS 
FIRST EMBODIMENT 



FIG. 3 shows a structure of an image defocusing apparatus in accordance with the first embodiment of 
the present invention. The image defocusing apparatus comprises an input unit 1 0, a display unit 20, a CG(com- 

30 puter graphics) processing unit 30, an image forming buffer 40, an original-image saving buffer 50, and a de- 
focus processing unit 100. Since the resulting defocused image is synthesized into a video taken by a video 
camera, lens's focusing effects must be taken into account at the time of defocusing. For this reason, data 
related to the optical characteristics of a lens being used, such as a zoom value, an iris value, a focal point, 
have been inputted into the defocus processing unit 100 in advance. The explanation of the structure and op- 

35 eration for the synthetic-image formation is omitted herein; for they are not the gist of the present invention. 

The input unit 1 0 is used to input data, such as geometric data, light source data, view point data, and image 
distance data into the CG processing unit 30. The details of the data are as follows: 
Geometric data : Shape of an object to be modeled 

Light source data : Location of a light source 

40 View point data : Location of a view point, a view vector, and a viewing angle 

Image distance data: Distance from the view point to an image 

The display unit 20 includes a display screen 21 , a frame memory 22, and a switching device 23. The dis- 
play screen 21 displays the data either in the frame memory 22 or the image forming buffer 40 under the control 
of the switching device 23 with a bank-switching command sent from the defocus processing unit 100. 

45 The CG processing unit 30 forms an original image of an object to be modeled by a process called "ren- 

dering" with the data from the input unit 10. The image forming process will be described in the following by 
referring to FIG. 4, although the explanation of the "rendering" is omitted. 

To begin with, the CG processing unit 30 forms an object 01 at a focal length in a boundary box with the 
geometric data. Then, the CG processing unit 30 fixes a view point C1 at (0, 0, 0) in x, y, z coordinates with 

50 the view point data, while placing a screen S1 with the image distance data; C1 corresponds to a location of 
the video camera, and S1 is an image plane for the original image of 01 . Accordingly, the CG processing unit 
30 computes a Z value of each pointfor their respective corresponding pixels; the Z value represents a distance 
between C1 and each point in the z coordinate, the direction in which the video camera is aimed. Thus, the Z 
values given to pixels G1, G3 are the z coordinates of a point G2 on 01 and a point G4 on the background, 

55 respectively. Further, the CG processing unit 30 removes hidden-surfaces using the Z values thus given, and 
outputs the resulting original image to the image forming buffer 40 first, and then saves the same in the orig- 
inal-image saving buffer 50. On the other hand, the CG processing unit 30 activates the defocus processing 
unit 100 by sending an operation-start command together with the Z values. 
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The image forming buffer 40 receives the original image and stores values representing RGB intensities 
on each pixel, i.e., pixel values, of the original image in relation with their x, y coordinates. For example, the 
original image of FIG. 4 is shown in FIG. 6A, and for the pixels within a rectangle d the image forming buffer 
40 stores values as shown in FIG. 6B. Each rectangle in the grid represents the pixels, while the values in 
5 brackets the RGB intensities, and the rows and columns the x, y coordinates, respectively. Note that the orig- 
inal pixel values are updated by the values from the defocus processing unit 100. 

The original-image saving buffer 50 saves the original image in the same way as the image forming buffer 

40. 

The defocus processing unit 100 forms a defocused image based on the original image with the Z values 
10 and optical characteristics of the lens. The structure of the defocus processing unit 100 is depicted in FIG. 5. 
The defocus processing unit 100 comprises a Z buffer 101, a COC(circle of confusion) calculator 102, a COC 
buffer 103, a Z comparator 104, a RP(reference pixel) address calculator 105, a pixel value calculator 106, 
and a controller 107. 

More precisely, the Z buffer 101 receives the Z value for each original pixel from the CG processing unit 
15 30 and stores them in relation with their x, y coordinates. For the rectangle d in FIG. 6 A, the Z buffer 101 stores 
the values as shown in FIG. 6C. 

The COC calculator 102 computes a radius C of a circle of confusion of each point for their corresponding 
pixels by Expression 1 as below with the Z value read out from the Z buffer 101 and the lens's optical char- 
acteristics data stored in advance. Note that the circle of confusion is an out-of-focus point appearing as a circle 
20 rather than a point in the image, and is a measure of how defocused the out-of-focus point is. 

[Expression 1] C = 1 1 - p(d - f)/d(p - f) I * f/F 
where p is a distance from the view point to the focal point, d is the Z value, f is the focal length, and F 
is the iris value. 

The COC buffer 103 stores the radiuses C of the circles of confusion in relation with their x, y coordinates. 

25 For the rectangle d in FIG. 6A, the COC buffer 102 stores the values as shown in FIG. 6D. 

The Z comparator 1 04 reads out the original pixels from the image forming buffer 40 one by one as a target 
pixel, and selects the neighboring pixels for each target pixel. The neighboring pixels are the pixels such that 
satisfy I X-u | ^t and I Y-v I ^t, given a target pixel(X,Y), an arbitrary pixel (u,v), and a constant t which varies 
with an amount of computation. Further, the Z comparator 104 reads out the Z values of both the target and 

30 neighboring pixels from the Z buffer 101 using their x, y coordinates to select the neighboring pixels whose Z 
values are smaller than the target pixel. The neighboring pixels thus selected are outputted to the RP address 
calculator 105 and referred to as nominated pixels hereinafter. 

The RP address calculator 105 computes a distance between the target pixel and each nominated pixel 
while reading out the radius C of each nominated pixel from the COC buffer 103 using their x, y coordinates, 

35 and selects the ones whose radiuses C are longer than the computed distance. The nominated pixels thus 
selected are outputted to the pixel value calculator 106 and referred to as reference pixels hereinafter. 

The selection of the reference pixels will be explained by using the rectangle d in FIG. 6Aas an example. 
FIG. 7A shows the pixel arrays within the rectangle d, where POO through P46 represent the original pixels. 
Assume that P23 is read out as the target pixel and POO is selected as one of the nominated pixels. Then, the 

40 RP address calculator 105 computes a distance between P23 and POO with their x, y coordinates, i.e. V3 2 + 
4 2 = 5, and accordingly compares the distance, 5, with the radius C of POO's circle of confusion, 0 as shown 
in FIG. 6D. Given that the radius C is shorter than the distance, the POO is left out from the nomination. In 
case of another nominated pixel P25, the distance is 2 while the radius C is 3 as shown in FIG. 6D, and thus, 
P25 is selected as the reference pixel. FIG. 7B shows a set of the reference pixels for P23 selected in this way. 

45 The pixel value calculator 1 06 computes the distributions of the original values of the reference pixels within 

their own circles of confusion and a weighted value of the target pixel to yield a mean value. Accordingly, the 
pixel value calculator 1 06 updates the original pixel values in the image forming buffer 40 with the mean values. 
More precisely, the pixel value calculator 106 computes an area of the circle of confusion for each reference 
pixel using their respective radiuses C, and reads out the original pixel values for the reference pixels from 

50 the image saving buffer 50. Accordingly, the pixel value calculator 106 multiplies the original pixel values with 
a weight coefficient, i.e., the reciprocal of the area of their own circles of confusion, to compute the distributions. 
Likewise, the pixel value calculator 106 computes an area of the target pixel's circle of confusion, and reads 
out the original pixel value of the target pixel from the image saving buffer 50 to multiply the original pixel value 
with the reciprocal. Note that when the target pixel is in focus(exhibiting a 0-radius), the pixel value calculator 

55 1 06 invariably gives a value 1 to the reciprocal. 

The controller 107 controls the above-described components by sending an execution command and re- 
ceiving an operation-completion signal. 

The operation of the above-described defocus processing unit 100 will be explained by referring to the 
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flowchart in FIG. 8. Note that the controller 107 activates the Z comparator 104 when it has received the op- 
eration-completion signal from the COC calculator 102, and likewise, the RP address calculator 1 05 from the 
Z comparator 104, the pixel value calculator 106 from the RP address calculator 105, and these descriptions 
are not repeated in the following. 

5 Upon the receipt of the operation-start command from the CG processing unit 30, the controller 107 ac- 

tivates the COC calculator 102. The COC calculator 1 02 then computes the radius C of the circle of confusion 
of each point in the boundary box for their respective pixels, and the computation results are stored into the 
COC buffer 103 in relation with their x, y coordinates(Steps 101-104). Subsequently, the controller 107 acti- 
vates the Z comparator 104, so that it reads out one original pixel as the target pixel from the image forming 

10 buffer 40 and selects its neighboring pixels while reading out their Z values from the Z buffer 101 to select 
the nominated pixels( Steps 105-106). Then, the controller 107 activates the RP address calculator 105, which 
correspondingly selects the reference pixels from the nominated pixels(Step 107). Accordingly, the controller 
107 activates the pixel value calculator 106, which, in response, computes the distributions of the reference 
pixels within their own circles of confusion and the weighted value of the target pixel(Step 108), and further 

15 computes the mean value to update the original value of the target pixel(Step 109). finally, the controller 107 
checks whether ail the original pixels in the image forming buffer 40 are read out as the target pixel or not(Step 
110). If so, it issues the bank-switching command to the switching unit 23, so that the resulting defocused image 
in the image forming buffer 40 Is displayed on the display screen 21 (Step 111); otherwise it returns to Step 
105. 

20 FIG. 9 is a subroutine of Step 107, detailing the operation of the RP address calculator 105. The RP address 

calculator 105 receives the nominated pixels for the target pixel from the Z comparator 104(Step 201), and 
computes a distance between the target pixel and each nominated pixel(Step 202). Accordingly, the RP ad- 
dress calculator 105 compares the computed distance and the radius C of the circle of confusion of each nom- 
inated pixel to select the reference pixels(Steps 203-204), and checks whether it has received all the nominated 

25 pixels for the target pixel or not(205). If not, it returns to Step 201; otherwise it terminates the operation and 
sends the operation-completion signal to the controller 107. 

From the foregoing description, it is understood that the points in focus do not appear in the circles of con- 
fusion, and thus their corresponding pixels are given the 0-radius, and left out from the nomination. Therefore, 
even when their corresponding pixels are overlapped by the circles of confusion of the neighboring out-of-focus 

50 points, their pixel values are not distributed within the overlapping circles of confusion. As a result, the points 
in focus appear clear and sharp while the out-of-focus points defocused correspondingly to their Z- values, ena- 
bling the object's edge to appear dear and sharp and in a size it should when the object is in focus. Also, in 
case of the out-of-focus points, their corresponding pixels whose Z values are larger than target pixel are not 
selected as the nominated pixels, and the nominated pixels whose radiuses C are shorter than the distance 

35 to the target pixel are left out from the nomination, enabling the pixel value distribution in accordance with their 
spatial location. Thus, the points both in focus and out of focus appear in the approximate way when seen 
through a camera. 

In the foregoing embodiment, the neighboring pixels having larger Z values than the target pixel are ex- 
cluded from the nominated pixels. However, they may be included in the reference pixels on the condition that 
40 the controller 1 07 disallows the pixel value calculator 106 to compute the distributions relative to these neigh- 
boring pixels. 

Note that the defocus processing unit 100 may include a plurality of pixel value calculators 106 for parallel 
processing. 

45 SECOND EMBODIMENT 

An image defocusing apparatus in accordance with the second embodiment of the present invention is 
identical with that of the first embodiment except that the defocus processing unit 100 is replaced with a de- 
focus processing unit 300. 

50 More precisely, as is shown in FIG. 10, the Z comparator 104 is replaced with a first Z comparator 304 

and a second Z comparator 308; the RP address calculator 105 is replaced with a first RP address calculator 

305 and a second RP address calculator 309; the pixel value calculator 106 with a first pixel value calculator 

306 and a second pixel value calculator 310; the controller 107 with a controller 311; and an IPV(intermediate 
pixel value) buffer 308 is additionally included. Hereinafter, like components are labeled with like reference 

55 numerals with respect to the first embodiment, and the description of these components is not repeated. 

The first Z comparator 304 operates in the same way as the Z comparator 104 except that it outputs the 
nominated pixels to the first RP address calculator 305. 

The first RP address calculator 305, unlike the RP address calculator 105, selects the reference pixels 
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within a certain range along the x coordinate; hence the reference pixels have radiuses C larger than the certain 
value. For example, when P03 in FIG. 7 is the target pixel, then the first RP address calculator 305 selects 
PCM, P05 as the reference pixels as is shown in FIG. 11A; when P04, then P03, P05 in FIG. 11B; when P05, 
then P03, P04 in FIG. 11C. The reference pixels thus selected are outputted to the first pixel value calculator 
5 305. 

The first pixel value calculator 306 computes the distributions of the reference pixels within their own cir- 
cles of confusion and the weighted value of the target pixel, and further the mean value in the same way as 
the pixel value calculator 106. The value thus calculated is referred to as an intermediate pixel value herein- 
after. 

10 The intermediate pixel value buffer 307 stores the intermediate pixel values in relation with their x, y co- 

ordinates. 

The second Z comparator 308 operates in the same way as the Z comparator 104 except that it outputs 
the nominated pixels to the second RP address calculator 309. 

The second RP address calculator 309 selects the reference pixels within a certain range along the y co- 
ys ordinate; hence the reference pixels have the radiuses C larger than the certain value. For example, when P03 
in FIG. 7 is the target pixel, then the second RP address calculator 309 selects P13, P23, and P33 as the ref- 
erence pixels as is shown in FIG. 12A; when P13, then P03, P23, and P33 in FIG. 12B; when P23, then P03, 
P13, P33 in FIG. 12C. The reference pixels thus selected are outputted to the second pixel value calculator 
310. 

20 The second pixel value calculator 310 computes the distributions of the reference pixels within their own 

circles of confusion and the weighted value of the target pixel using the intermediate pixel values read out from 
the lPV buffer 307 in the same way as the pixel value calculator 106, and further computes the mean value. 
The value thus computed is referred to as the defocused pixel value hereinafter. 

The controller 311 controls above-described components by sending the execution command and receiv- 
25 ing the operation-completion signal in the same way as the controller 107. 

The operation of the above-described defocus processing unit 300 will be explained by referring to the 
flowchart in FIG. 13. The explanation for Steps 101 through 104 is omitted; for they are identical with FIG. 8 
in the first embodiment. 

Having received the notice of operation-completion signal from the COC calculator 102, the controller 311 

30 activates the first Z comparator 304, which, in response, reads out one original pixel from the image forming 
buffer 40 and selects the neighboring pixels to further select the nominated pixels(Steps 305, 306). 

Subsequently, the controller 311 activates the first RP address calculator 305, which then selects the ref- 
erence pixels from the nominated pixels relative to the value in the x-coordinate(Step 307). Next, the controller 
311 activates the first pixel value calculator 305, which computes the distributions of the reference pixels within 

35 their own circles of confusion and the weighted value of the target pixel, and further the mean value, i.e., the 
intermediate pixel value(Steps 308, 309). Then, the controller 311 checks whether all the pixels in the image 
forming buffer 40 are read out as the target pixel or not(Step 310). If not, it returns to Step 305; otherwise it 
activates the second Z comparator 308, which, in response, selects the neighboring pixels to further select 
the nominated pixels(Steps 311, 312). Subsequently, the controller 311 activates the second RP address cal- 

40 culator 310, which accordingly selects the reference pixels from the nominated pixels relative to the value in 
the y coord in ate(Step 313). Subsequently, the controller 311 activates the second pixel value calculator 310, 
so that it computes the distributions of the reference pixels within their own circles of confusion and the weight- 
ed value of the target pixel by using the intermediate pixel values, and further the mean value, i.e., the dis- 
tributed pixel value(Steps 314, 315). Finally, the controller 311 checks whether all the pixels in the image form- 

45 ing buffer 40 are read out as the target pixel(Step 316). If not, it returns to Step 311; otherwise, it issues the 
bank-switching command to the switching unit 23, so that the resulting defocused image in the image forming 
buffer 40 is displayed on the display screen 21 (Step 111) 

By using the x, y coordinates separately, the reference pixels are selected faster than the first embodiment; 
the distance from the target pixel to each nominated pixel can be computed by simply counting the number of 

so pixels between the two pixels in respective coordinates, instead of computing the root-square-mean value. 

Note that the first RP address calculator 305 may select the reference pixels relative to the value in y- 
coordinate, and the second RP address calculator 309 x-coordinate. 

In the foregoing embodiment, the neighboring pixels having larger Z values than the target pixel are ex- 
cluded from the nominated pixels. However, they may be included in the reference pixels on the condition that 

55 the controller 311 disallows the first and second pixel value calculators 306, 310 to compute the distributions 
relative to these neighboring pixels. 

Although the present invention has been fully described by way of example with reference to the accom- 
panying drawings, it is to be noted that various changes and modification will be apparent to those skilled in 
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the art. Therefore, unless otherwise such changes and modifications depart from the scope of the present 
invention, they should be construed as being included therein. 



5 Claims 

1. An image defocusing apparatus for defocusing an image formed on a screen by modeling an object in a 
boundary box by updating pixel values on the image to approximate a lens's focusing effects, the appa- 
ratus comprising: 

10 means for computing data related to a circle of confusion for each point in the boundary box, the 

circle of confusion being a measure how defocused an out-of-focus point is on the screen; 

means for checking a reference pixel whose corresponding point's circle of confusion overlaps a 
target pixel; and 

means for updating an original value of the target pixel by taking into account a value of the refer- 
15 ence pixel and the data related to the target pixel's circle of confusion. 

2. The apparatus of Claim 1, wherein the data related to the circle of confusion comprises a radius and an 
area of each circle of confusion, and 

wherein the circle-of-confusion computing means includes: 
20 a radius calculating unit for computing a radius C of each circle of confusion using an expression: 

C = |1 -p(d-f)/d(p-f)| * f/F 
where p is a distance from a view point to a focal point, d is a Z value that represents a distance 
from the view point to each point, f is a focal length, and F is an iris value of a lens being used, and 
wherein the updating means includes: 
25 an area calculating unit for computing an area of each circle of confusion using their respective 

radiuses C. 

3. The apparatus of Claim 2, wherein the circle-of-confusion computing means further includes a storage 
unit for storing the radiuses C of the circles of confusion in relation with their respective pixels' x, y coor- 

30 dinates. 

4. The apparatus of Claim 3, wherein the checking means includes: 

a distance computing unit for computing a linear distance from the target pixel to each neighboring 

pixel; 

a comparing unit for comparing the linear distance with the radius C relative to each neighboring 
pixel by referring to their x, y coordinates; and 

a comparison-result outputting means for selecting a neighboring pixel whose radius C is longer 
than the linear distance, and subsequently outputting the selected neighboring pixel as the reference pix- 
el. 

5. The apparatus of Claim 4, wherein the updating means includes: 
a distributing unit for weighting a value of each reference pixel to determine a distribution within 

their own circles of confusion, the distribution being distributed to the target pixel; 

a target-pixel weighting unit for weighting the original value of the target pixel with the area of its 
circle of confusion; and 

an updating unit for accumulating the distributions and the weighted value of the target pixel to 
yield a mean value, the original value of the target pixel being updated by the mean value. 

6. The apparatus of Claim 5, wherein the distributing unit includes a reference- pixel weighting unit for mul- 
tiplying a value of each reference pixel by a reciprocal of an area of their respective circles of confusion. 

50 

7. The apparatus of Claim 5, wherein the checking means further includes: 

a spatial-location comparing unit for comparing Z values of the target pixel and the reference pixels, 
the Z value representing a distance from the view point to their corresponding points, and 

wherein the image defocusing apparatus further comprising: 
55 means for disallowing the updating means to update the original value of the target pixel when the 

reference pixel has a larger Z value than the target pixel. 

8. The apparatus of Claim 5, wherein the checking means further includes: 
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an output unit for comparing the Z values of the target pixel and the neighboring pixels to select a 
neighboring pixel whose Z value is smaller than the target pixel, and for subsequently outputting the se- 
lected neighboring pixel to the distance computing unit, the Z value representing a distance from the view 
point to their corresponding points. 

An image defocusing apparatus for defocusing an image formed on a screen by modeling an object in a 
boundary box by updating pixel values on the image to approximate a lens's focusing effects, the appa- 
ratus comprising: 

means for computing data related to a circle of confusion for each point in the boundary box, the 
circle of confusion being a measure how defocused an out-of- focus point is on the screen; 

a first checking means for checking a first reference pixel whose corresponding point's circle of 
confusion overlaps a target pixel in a first direction; 

a first updating means for updating an original value of the target pixel by taking into account a 
value of the first reference pixel and the data related to the target pixel's circle of confusion; 

a second checking means for checking a second reference pixel whose corresponding point's circle 
of confusion overlaps the target pixel in a second direction, the second direction being orthogonal with 
respect to the first direction; and 

a second updating means for updating the values updated by the first updating unit by taking into 
account a value of the second reference pixel and the data related to the target pixel's circle of confusion. 

The apparatus of Claim 9, wherein the data related to the circle of confusion comprises a radius and an 
area of each circle of confusion, and 

wherein the circle-of-confusion computing means includes: 

a radius calculating unit for computing a radius C of each circle of confusion using an expression: 

C = 1 1 - p(d - fyd(P - f) I * f/F 
where p is a distance from a view point to a focal point, d is a Z value that represents a distance 
from the view point to each point, f is a focal length, and F is an iris value of a lens being used, and 
wherein the updating means includes: 

an area calculating unit for computing an area of each circle of confusion using their respective 
radiuses C. 

The apparatus of Claim 10, wherein the circle-of-confusion computing means further includes a storage 
unit for storing the radiuses C of the circles of confusion in relation with their respective pixels' x, y coor- 
dinates. 

The apparatus of Claim 11, wherein the first checking means includes: 

a first distance computing unit for counting the number of pixels between the target pixel and each 
pixel aligned in an array that contains the target pixel in the first direction, the number thus counted being 
a distance; 

a first comparing unit for comparing the distance and the radius C relative to each pixel aligned in 
the target pixel's array in the first direction by referring to their x, y coordinates; 

a first comparison-result outputting unit for outputting a pixel aligned in the first direction whose 
radius is longer than the distance to the first updating means as the first reference pixel, and 

wherein the second checking means includes: 

a second distance computing unit for counting the number of pixels between the target pixel and 
each pixel aligned in an array that contains the target pixel in the second direction, the number thus count- 
ed being a distance; 

a second comparing unit for comparing the distance and the radius C relative to each pixel aligned 
in the target pixel's array in the second direction by referring to their x, y coordinates; and 

a second comparison-result outputting unit for outputting a pixel aligned in the second direction 
whose radius is longer than the distance to the second updating means as the second reference pixel. 

The apparatus of Claim 12, wherein the first updating means includes: 

a first distributing unit for weighting a value of the first reference pixel to determine a distribution 
within their own circles of confusion, the distribution being distributed to the target pixel, and 

wherein the second updating means includes: 

a second distributing unit for weighting a value of the second reference pixel to determine a dis- 
tribution within their own circles of confusion, the distribution being distributed to the target pixel. 
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14. The apparatus of Claim 13, wherein the first distributing unit includes a first reference-pixel weighting 
unit for multiplying an original value of each reference pixel by a reciprocal of an area of their respective 
circles of confusion, and the second distributing unit includes a second reference-pixel weighting unit for 
multiplying an original value of each reference pixel by a reciprocal of an area of their respective circles 

5 of confusion. 

1 5. The apparatus of Claim 1 4, further comprising: 

means for withholding the value of the target pixel updated by the first updating means to transfer 
them to the second updating means. 

10 

1 6. The apparatus of Claim 1 3, wherein the first checking means includes: 

a first spatial-location comparing unit for comparing 2 values of the target pixel and the first ref- 
erence pixel, the Z value representing a distance from the view point to their corresponding points, and 

wherein the second checking unit includes: 
15 a second spatial-location comparing means for comparing the Z values of the target pixel and the 

second reference pixel, and 

wherein the first updating means include; 

a first disallowing unit for disallowing the first updating means to update the original value of the 
target pixel when the target pixel has a smaller Z value, and 
20 wherein the second updating means include: 

a second disallowing unit for disallowing the second updating means to update the original value 
of the target pixel when the target pixel has a smaller Z value. 
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17. The apparatus of claim 13, wherein the first checking means further includes: 

a first output unit for comparing the Z values of the target pixel and the neighboring pixels to select 
a neighboring pixel whose Z value is smaller than the target pixel, and for subsequently outputting the 
selected neighboring pixel to the first distance computing unit, the Z value representing a distance from 
the view point to their corresponding points, and 

wherein the second checking means further includes: 

a second output unit for comparing the Z values of the target pixel and the neighboring pixels to 
select a neighboring pixel whose Z value is smaller than the target pixel, and for subsequently outputting 
the selected neighboring pixel to the second distance computing unit. 

18. A method of defocusing an image by updating pixel values on the image formed on a screen by modeling 
an object in a boundary box, the method comprising the steps of: 

(1) computing data related to a circle of confusion for each point in the boundary box, the circle of con- 
fusion being a measure how defocused an out-of-focus point is on the screen; 

(2) checking a reference pixel whose corresponding point's circle of confusion overlaps a target pixel; 
and 

(3) updating an original value of the target pixel by taking into account a value of the reference pixel 
and the data related to the target pixel's circle of confusion. 



19. The method of Claim 18, wherein the first step comprises the substep of: 

(a) computing a radius C of each circle of confusion using an expression: 

C = 1 1 - p(d - f)/d(p - f) I * f/F 
45 where p is a distance from a view point to a focal point, d is a Z value that represents a distance 

from the view point to each point, f is a focal length, and F is an iris value of a lens being used. 

20. The method of Claim 19, wherein the second step comprises the sub-steps of: 

(a) computing a distance from the target pixel to each neighboring pixel; 
50 (b) comparing the distance with a radius C of the overlapping circle of confusion relative to each neigh- 

boring pixel; 

(c) selecting a neighboring pixel whose radius C is longer than the distance as the reference pixel; 

(d) selecting another target pixel; and 

(e) proceeding the third step when all the pixels on the image have been selected as the target pixel 
55 and been through the substeps (a) through (c) of the second step. 

21. The method of Claim 20, wherein the third step comprises the substeps of: 

(a) weighting a value of each reference pixel to determine a distribution to the target pixel; 
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(b) computing an area of each overlapping circle of confusion and the target pixel's circle of confusion 
to weight the original value of the target pixel with the area of its circle of confusion; 

(c) accumulating thedistributions and the weighted value of the target pixel to yield a mean value, and 
updating the original value of the target pixel with the mean value; 

5 (d) selecting another target pixel; and 

(e) terminating to update the target pixel's value when all the pixels have been selected as the target 
pixel and been through the substeps (a) through (c) of the third step. 

22. The method of Claim 21 further comprising the steps of: 
10 (4) comparing Z values of the target pixels and neighboring pixels, the 2 value representing a distance 

from a view point to their corresponding points, and 

(5) disallowing to update the original value of the target pixel when the neighboring pixel has a larger 
Z value than the target pixel. 

15 23. The method of Claim 21 , wherein the second step further comprises the substep of: 

(f) comparing Z values of the target pixels and neighboring pixels to select a neighboring pixel 
whose Z value is smaller than the target pixel prior to the substep (a) of the second step, the Z value rep- 
resenting a distance from a view point to their corresponding points. 

2o 24. A method of defocusing an image by updating pixel values on the image formed on a screen by modeling 
an object in a boundary box, the method comprising the steps of: 

(1) computing data related to a circle of confusion for each point in the boundary box, the circle of con- 
fusion being a measure how defocused an out-of-focus point is on the screen; 

(2) checking a first reference pixel whose corresponding point's circle of confusion overlaps a target 
pixel along a first direction; 

(3) updating an original value of the target pixel by taking into account a value of the first reference 
pixel and the data related to the target pixel's circle of confusion; 

(4) checking a second reference pixel whose corresponding point's circle of confusion overlaps the tar- 
get pixel along a second direction, the second direction being orthogonal with respect to the first di- 
rection; and 

(5) updating the value updated at the third step by taking into account a value of the second reference 
pixel and the data related to the target pixel's circle of confusion. 
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25. The method of Claim 24, wherein the second and fourth steps respectively comprise the substep of: 

(a) comparing Z values of the target pixels and neighboring pixels, the Z value representing a dis- 
tance from a view point to their corresponding points, and 

wherein the third and fifth steps respectively comprise the substep of: 

(a) disallowing to update the original value of the target pixel when the neighboring pixel has a larger 
Z value than the target pixel. 

40 26. The method of Claim 25, wherein the second and fourth steps further and respectively comprise the sub- 
step of: 

(b) comparing the Z values of the target pixels and neighboring pixels to select a neighboring pixel 
whose Z value is smaller than the target pixel prior to the substep (a) of the second step and the substep 
(a) of the fourth step, respectively. 
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FIG. 3 
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FIG. 6A 
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